20 research outputs found
Temporal coherence-based self-supervised learning for laparoscopic workflow analysis
In order to provide the right type of assistance at the right time,
computer-assisted surgery systems need context awareness. To achieve this,
methods for surgical workflow analysis are crucial. Currently, convolutional
neural networks provide the best performance for video-based workflow analysis
tasks. For training such networks, large amounts of annotated data are
necessary. However, collecting a sufficient amount of data is often costly,
time-consuming, and not always feasible. In this paper, we address this problem
by presenting and comparing different approaches for self-supervised
pretraining of neural networks on unlabeled laparoscopic videos using temporal
coherence. We evaluate our pretrained networks on Cholec80, a publicly
available dataset for surgical phase segmentation, on which a maximum F1 score
of 84.6 was reached. Furthermore, we were able to achieve an increase of the F1
score of up to 10 points when compared to a non-pretrained neural network.Comment: Accepted at the Workshop on Context-Aware Operating Theaters (OR
2.0), a MICCAI satellite even
AutoSNAP: Automatically Learning Neural Architectures for Instrument Pose Estimation
Despite recent successes, the advances in Deep Learning have not yet been
fully translated to Computer Assisted Intervention (CAI) problems such as pose
estimation of surgical instruments. Currently, neural architectures for
classification and segmentation tasks are adopted ignoring significant
discrepancies between CAI and these tasks. We propose an automatic framework
(AutoSNAP) for instrument pose estimation problems, which discovers and learns
the architectures for neural networks. We introduce 1)~an efficient testing
environment for pose estimation, 2)~a powerful architecture representation
based on novel Symbolic Neural Architecture Patterns (SNAPs), and 3)~an
optimization of the architecture using an efficient search scheme. Using
AutoSNAP, we discover an improved architecture (SNAPNet) which outperforms both
the hand-engineered i3PosNet and the state-of-the-art architecture search
method DARTS.Comment: Accepted at MICCAI 2020 Preparing code for release at
https://github.com/MECLabTUDA/AutoSNA
TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks
Automatic surgical phase recognition is a challenging and crucial task with
the potential to improve patient safety and become an integral part of
intra-operative decision-support systems. In this paper, we propose, for the
first time in workflow analysis, a Multi-Stage Temporal Convolutional Network
(MS-TCN) that performs hierarchical prediction refinement for surgical phase
recognition. Causal, dilated convolutions allow for a large receptive field and
online inference with smooth predictions even during ambiguous transitions. Our
method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy
videos with and without the use of additional surgical tool information.
Outperforming various state-of-the-art LSTM approaches, we verify the
suitability of the proposed causal MS-TCN for surgical phase recognition.Comment: 10 pages, 2 figure
Endo-Sim2Real: Consistency Learning-Based Domain Adaptation for Instrument Segmentation
Surgical tool segmentation in endoscopic videos is an
important component of computer assisted interventions
systems. Recent success of image-based solutions using
fully-supervised deep learning approaches can be attributed
to the collection of big labeled datasets. However, the
annotation of a big dataset of real videos can be prohibitively
expensive and time consuming. Computer simulations could
alleviate the manual labeling problem, however, models
trained on simulated data do not generalize to real data. This
work proposes a consistency-based framework for joint
learning of simulated and real (unlabeled) endoscopic data
to bridge this performance generalization issue. Empirical
results on two data sets (15 videos of the Cholec80 and
EndoVis’15 dataset) highlight the effectiveness of the
proposed Endo-Sim2Real method for instrument
segmentation. We compare the segmentation of the
proposed approach with state-of-the-art solutions and show
that our method improves segmentation both in terms of
quality and quantity